Cluster-based architectures are a useful platform for hosting many Internet applications such as Web servers or Web services. A cluster-based system includes a front-end (gateway) node connected by a local area network to a set of back-end nodes. The front-end node receives the requests and forwards it to a back-end node where the actual processing of the request takes place. There are many advantages to a cluster based system, including incremental scalability, increased availability and performance, cost control and maintenance. Hence, clusters are used for scalable Web servers, as described by M. Aron, D. Sanders, P. Druschel, and W. Zwaenepoel, in “Scalable Content-Aware Request Distribution in Cluster-based Network Servers”, Proceedings of 2000 USENIX Annual Technical Conference, June 2000, and described by M. Aron, P. Druschel and W. Zwaenepoel in “Efficient Support for P-HTTP in Cluster-based Web Servers”, Proceedings of 1999 USENIX Annual Technical Conference, June 1999.
A specific technique for content-based request distribution within cluster-based network servers is described by V. S. Pai, M. Aron, G. Banga, M. Svendsen, P. Druschel, W. Zwaenepoel, and E. Nahum in “Locality-Aware Request Distribution in Cluster-based Network Servers”, Proceedings of 8th ACM Conference on Architectural Support for Programming Languages and Operating Systems, October 1998. Locality-aware request distribution (LARD) involves dividing data into partitions on the back-end servers and using a front-end to distribute incoming requests in a manner that takes account of where the data is stored.
A cluster-based architecture is also suitable for the provision of Web services, as described by E. Casalicchio and M. Colajanni in “A Client-Aware Dispatching Algorithm for Web Clusters Providing Multiple Services”, Proceedings of the 10th International World Wide Web Conference, May 2001. The advantages of clusters for Internet service provision are described by A. Fox, S. Gribble, Y. Chawathe, E. Brewer and P. Gauthier in “Cluster-Based Scalable Network Services”, Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles, October 1997.
M. Welsh, D. Culler and E. Brewer, in “SEDA: An Architecture for Well-Conditioned, Scalable Internet Services”, Proceedings of 18th Symposium on Operating Systems Principles (SOSP '01), October 2001, describe a framework for highly concurrent server applications which uses threading and aspects of event-based programming models to provide automatic tuning in response to large variations in load. The framework is referred to as the staged event-driven architecture (SEDA). Stages are separated by event queues and include controllers which dynamically adjust resource allocation and scheduling in response to changes in load. The size of each stage's thread pool is adjusted based on the monitored length of an event queue.
Various mechanisms have been tried to improve the performance of cluster based systems—including caching, load balancing and client hand-off (e.g. TCP hand-off). The front-end may distribute the requests such that the load among the back-end nodes is balanced and the load may be distributed based on the client, request content, current resource usage or scheduling algorithms. Load distribution based on clusters is known to improve the scalability, availability and fault tolerance of Internet services. Various hand-off mechanisms have been reported in literature for request forwarding. However these typically require changes in the operating system and are not portable.
In “On the Use of Virtual Channels in Networks of Workstations with Irregular Topology”, IEEE Transactions on Parallel and Distributed Systems, Vol. 11, No. 8, August 2000, pages 813-828, F. Silla and J. Duato describe a flow control protocol for supporting a number of virtual channels in a network of workstations which implements adaptive routing. The flow control protocol uses channel pipelining and seeks to minimize control traffic.
Published US Patent Application Nos. 2002/0055980, 2002/0055982 and 2002/0055983 (Goddard) describe a server computer, such as a cluster-based Web server, having a plurality of persistent connections to a dispatcher. The dispatcher monitors the performance of the back-end server and, in order to improve back-end server performance, controls either the number of concurrently processed data requests or the number of concurrently supported connections.
Although a lot of work has been done to improve the performance of cluster-based systems, there is scope for further improvement. For example, the existing cluster-based systems do not make effective use of the network bandwidth between the front-end and back-end nodes. Existing cluster-based systems do not fully exploit the benefits of multiple connections. The existing systems are generally configured statically and do not adapt to the changing workload on the system. Further, the existing systems are generally based on direct network subsystems (e.g. TCP/IP) and do not exploit the benefits of the mediated network subsystems (e.g. Java Messaging Service).